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THE  PROGRAM  MANAGER’S  ASSOCIATE 


INTRODUCTION 


Objectives 

The  purpose  of  this  Small  Business  Technology  Transfer  Program  (STTR)  project  is  to  define 
requirements  for,  design,  and  prototype  a  computer-based  support  system  for  R&D  program 
managers  (PMs).  The  long-range  goal  is  to  develop  an  architecture  and  set  of  tools  with  appeal 
to  PMs  independently  of  their  organization  (government,  industry,  academia)  and  domain  (e.g., 
materials  sciences  or  applied  behavioral  sciences;  fundamental  research  or  advanced 
development). 


OVERVIEW 

This  report  presents  the  final  project  status.  First,  we  present  our  approach  to  determine 
PMA  functional  requirements.  Next,  we  share  our  design  for  the  PMA  to  support  those 
requirements.  Then,  we  indicate  the  specific  support  tools  and  briefly  explore  the  potential  of  the 
PMA.  This  final  report  is  accompanied  by  a  demonstration  of  the  PMA  software. 


Approach 

Our  approach  to  develop  support  system  requirements  involved  three  steps.  First,  we 
developed  and  administered  an  interview  protocol  to  ARPA  program  managers  that  focused  on 
the  nature  of  their  jobs,  their  information  needs,  and  difficulties  they  may  have  experienced  in 
becoming  “ARPA-proficient”  as  new  managers.  Interviewees  were  Craig  Wier  and  Bob  Neches 
(two  occasions  each),  and  Pradeep  Khosla  (current  PMs),  and  Robert  Simpson  and  Steve  Cross 
(former  Pms).  Mike  Kelly  also  provided  advice.  Interviews  lasted  froml  to  2.5  hours. 

Next,  we  analyzed  the  results  of  these  interviews,  seeking  difficulties  and  needs  that  were 
common  across  managers.  This  analysis  led  to  a  descriptive  framework  which  we  used  to  specify 
PMA  functions.  We  also  compared  the  difficulties  and  needs  of  ARPA  PMs  with  those  we 
identified  using  a  similar  approach  among  PMs  at  SEMATECH,  the  ARPA-sponsored  R&D 
consortium  for  the  semiconductor  industry.  We  expected  that  similarities  across  these  agencies 
would  help  us  specify  an  adaptable,  general  R&D  program  manager’s  support  system. 

We  then  identified  the  support  functions  that  would  aid  PMs  and  configured  these  into  a 
system  architecture  based  on  object-oriented  technology. 

The  intent  was  to  devise  a  general  application  that  would  reside  on  the  PM’s  desktop 
computer.  It  had  to  work  with  other  applications  that  PMs  currently  use,  yet  also  provide  a 
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platform  on  which  to  grow.  In  particular,  we  perceive  growth  potential  in  intelligent  aiding  to 
help  the  PM  access  information  from  varied  sources,  interpret  that  information,  and  maintain 
control  over  it. 

We  then  programmed  a  prototype  PMA  using  Power  Builder  to  demonstrate  the  overall 
application  and  many  of  its  initial  set  of  tools.  The  accompanying  demonstration  is  sufficiently 
functional  for  ARPA  to  assess  PMA’s  potential,  suggest  refinements,  and  determine  whether 
continued  development  is  in  order. 


FINDINGS 


The  ARPA  Environment 

ARPA  is  the  premier  R&D  organization  for  DoD.  Its  mission  is  to  help  maintain  U.S. 
technological  superiority  over,  and  to  prevent  technological  surprise  by,  its  potential  adversaries. 
The  agency’s  goal  is  to  pursue  as  many  imaginative  and  innovative  research  ideas  as  budget  and 
other  constraints  allow.  Furthermore,  with  the  change  from  DARPA  to  ARPA,  there  is  a  new 
emphasis  on  civilian  needs,  such  as  health  care,  and  on  transferring  technology  to  the  commercial 
sector.  There  is  now  pressure,  as  well,  to  evaluate  R&D  programs  throughout  the  Federal 
government. 

Relative  to  its  mission,  ARPA  is  not  a  large  agency  in  terms  of  staff  size.  It  employs  about 
180  people  and  has  very  few  layers  of  management.  Moreover,  the  tenure  of  the  individual  PM  at 
ARPA  typically  lasts  four  or  fewer  years.  These  characteristics  create  a  demanding  job  for  the 
PM. 

There  are  several  contributing  factors  to  note: 

1.  ARPA’s  mission  requires  the  PM  to  identify  and  invest  in  technologies  that  possess 
enormous  potential,  but  harbor  significant  risk  of  lost  investment. 

2.  Each  PM’s  “portfolio”  of  R&D  investments  tends  to  be  large,  representing  a  variety  of 
technologies,  requiring  integration  by  the  PM  to  realize  full  benefits. 

3.  Information  must  be  gathered  and  integrated  from  diverse  personal  and  other  sources. 

4.  Rapid  PM  proficiency  is  critical  due  to  the  heavy  responsibilities  and  short  tenure; 
shortage  of  mentoring  exacerbates  this. 


R&D  Program  Manager’s  Job 

As  Depicted  in  Figure  1,  the  Arpa  Program  Manager’s  Job  Involves  Four  Major 
Responsibilities: 
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Figure  1.  A  breakdown  of  the  R&D  program  manager’s  job 

Situation  assessment  refers  to  activities  that  keep  the  PM  abreast  of  the  current  state  and 
trends  in  two  broad  areas:  (1)  DoD  and  civilian  needs;  and  (2)  technological  capabilities  and 
trends  that  may  pertain  to  the  needs.  One  of  the  most  important  services  that  R&D  PMs  perform 
is  the  conceptual  marriage  of  these  needs  and  capabilities  -  “I  see  an  application  of  this 
technology  to  the  following  operational  problem....” 

Managers  keep  abreast  of  needs  and  technology  through  a  variety  of  channels.  Regarding 
needs,  broad  direction  comes  from  national  goals  for  defense,  education,  manufacturing,  etc.,  that 
are  established  by  the  White  House,  Congress,  and  the  Secretary  of  Defense.  More  detailed 
agendas  emerge  from  sponsors  inside  of  DoD,  the  intelligence  community,  and  other 
governmental  agencies.  At  these  levels,  potential  sponsors  seek  specific  operational  capabilities 
usually  targeted  at  long-range  plans.  Needs  emerge  in  briefings,  policy  statements,  strategic 
planning  documents,  and  descriptions  of  ongoing  program.  ARPA  PMs  also  gain  knowledge 
about  needs  when  they  receive  feedback  from  other  potential  sponsors  to  proposed  R&D 
programs. 

Regarding  technological  capabilities  and  trends,  interviewees  said  that  they  constantly  scan  the 
technology  landscape  for  new,  cutting  edge  developments.  They  do  this  mostly  via  in-person 
meetings,  especially  site  visits  to  existing  contractors,  professional  meetings  and  workshops.  The 
interviewees  also  said  that  they  keep  in  weekly  and  monthly  touch  with  their  principal 


5 


investigators  (Pis).  Due  to  ARPA’s  prominence,  these  Pis  often  include  some  of  the  world’s 
leading  authorities  and  researchers  in  a  given  domain.  Indeed,  more  than  one  interviewee  said 
that  die  breadth  of  the  Pi’s  knowledge  of  new  technologies  and  his  or  her  willingness  to  keep  the 
PM  informed  about  them  was  vital. 

Another  important  channel  for  keeping  up  to  speed  about  technology  is  via  solicited  and 
unsolicited  proposals  in  which  the  latest  technology  and  new  ideas  are  presented.  Technical 
updates  from  their  own  programs  represent  a  third  channel.  Several  managers  said  that  the  World 
Wide  Web  has  also  become  a  useful  resource  to  them.  Fourth  is  the  technical  and  scientific 
literature,  and  electronic  databases  abstracting  that  literature.  The  ARPA  PMs  we  spoke  with 
said  that  they  spent  very  little  time  searching  for  and  reading  abstracts  and  technical  articles  - 
basically,  the  channel  is  too  inefficient. 

Program  planning  and  marketing  include  tasks  that  convert  neat  ideas  into  viable  programs. 
The  PM  first  works  to  envision  a  future  operational  state  based  on  some  technological  innovation, 
and  then  tries  to  drum  up  enthusiasm  for  this  future  state  among  prospective  sponsors.  With  the 
breadth  of  most  ARPA  programs,  this  work  includes  many  different  stakeholders.  As  a  result, 
ARPA  PMs  said  that  they  spend  a  great  deal  of  time  trying  to  gain  endorsement  for  their  program 
ideas,  whether  these  ideas  are  brand  new  or  significant  expansions  to  existing  work  —  thus,  the  use 
of  the  term  “marketing”  in  this  component  of  the  R&D  manager’s  job. 

Planning  entails  judging  the  risks  associated  with  a  new  technology  and  its  likelihood  of 
having  genuine  operational  impact.  It  also  involves  decomposing  a  technology  into  its 
contributing  components,  and  creating  a  program  roadmap  that  allocates  time  and  resources  to 
component  development.  This  activity  entails  lots  of  “what  if’  analyses  as  PMs  try  to  balance 
resource  needs  and  amounts  of  time  to  develop  a  technology  with  the  interests  of  the  operational 
community.  Managers  appear  to  have  few  tools  to  support  these  analyses  and  judgments. 

Marketing  involves  persuasion  -  persuading  sponsors  to  endorse  the  project  from  an 
operational  perspective,  and  often  persuading  contractors  to  couch  innovation  in  the  correct 
operational  terms.  This  requires  many  in-person  presentations  to  potential  sponsors,  many  of 
whom  are  in  the  DoD  establishment,  and  to  prospective  contractors  via  BAA  announcements  and 
“road  shows.”  These  presentations  focus  on  five  key  questions: 

1.  What  is  this  proposed  program  attempting  to  accomplish  (operational  needs  or 
opportunity)? 

2.  How  are  the  needs  that  this  program  hopes  to  meet  being  met  today? 

3.  What’s  new  about  the  proposed  approach/technology? 

4.  What  are  the  criteria  by  which  program  success  should  be  measured? 

5.  What  are  the  resource  requirements? 

The  PMs  said  that  one  of  the  most  difficult  planning  and  marketing  tasks  is  conveying  to 
contractors  the  “scorecard”  by  which  research  program  outputs  will  be  evaluated.  Often 
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contractors  appear  insensitive  to  the  operational  needs  of  the  ultimate  sponsors  and  beneficiaries 
of  their  work,  often  focusing  exclusively  on  the  scientific  or  technological  merits  of  their  work. 
This  motivational  tension  jeopardizes  technology  transfer  into  practice.  One  PM  described  his 
programs  as  “unfortunately,  providing  hothouses  for  orchids  in  the  arctic.  Take  away  the 
insulating  properties  of  ARPA,  and  the  contractor’s  outputs  fail  to  thrive  in  the  operational 
marketplace.” 

Marketing  also  involves  keeping  program  stakeholders  informed  and  enthused  about  the 
effort,  a  difficult  task  over  the  life  and  evolution  of  a  program,  especially  in  the  very  dynamic 
technological  areas  in  which  ARPA  participates. 

Finally,  along  with  conceptual  program  planning,  ARPA  PMs  spend  a  significant  amount  of 
time  at  the  administrative  effort  of  putting  a  new  program  in  place.  This  involves  finding 
contracting  channels,  preparing  and  managing  formal  documentation  including  briefing  materials, 
RFIs  and  RFPs;  collecting,  managing  and  responding  to  proposals;  writing  SOWs,  and  so  forth. 

Program  monitoring  and  management  are  dominated  by  management  problem  solving, 
coordinating,  budget  management,  reporting  to  ARPA  management  and  communicating  with 
contractors.  Most  ARPA  PMs  manage  many  contracts  simultaneously.  Under  these 
circumstances,  they  said  that  two  tasks  were  most  troublesome  during  program  execution:  finding 
out  from  contractors  the  true  state  of  their  progress,  and  managing  the  schedules,  scopes  and 
resources  across  all  programs  simultaneously. 

In  addition,  the  PMs  find  financial  management  of  their  programs  to  be  a  problem.  ARPA’s 
financial  reporting  system  does  not  provide  PMs  with  up-to-the-minute  actual  and  accrued 
expenditures  per  expense  category,  per  program.  Instead,  reports  are  delivered  monthly,  reflect 
invoiced  and  paid  expenses,  and  do  not  reflect  accrued  work.  Hence,  PMs  keep  separate  account 
information  (e.g.,  in  personally  developed  Excel  spreadsheets),  to  track  resource  levels. 

Knowing  the  levels  (allocated  to  the  program,  obligated  but  not  spent,  accrued  and  paid)  is  crucial 
when  the  manager  must  re-allocate  monies  based  on  technical  progress,  new  opportunities,  or 
changed  constraints  (e.g.,  a  reduction  in  6.1  funding).  As  a  result,  each  PM  said  that  a  PM 
support  system  must  integrate  program  planning  with  program  execution,  especially  as  regards 
resources.  They  also  said  that  a  system  which  reminds  them  about  critical  dates  in  the  funding 
cycle  (e.g.,  when  funds  had  to  be  committed  to  avoid  losing  them  in  a  sweep)  would  be  very 
helpful. 

Communications  present  an  ongoing  requirement  to  keep  ARPA  management,  sponsors, 
contractors,  and  the  broader  technical  and  scientific  communities  informed  about  program 
progress.  Since  ARPA  has  very  little  resident  administrative  support,  PMs  spend  a  great  deal  of 
time  trading  messages  with  individuals  in  each  of  these  groups.  Hundreds  of  messages  per  PM 
may  be  traded  per  day,  an  enormous  communications  load  that  dominates  the  PM’s  time. 

Preparing  presentations  for  various  purposes  also  consumes  precious  PM  resources.  Such 
preparations  may  entail  searching  for  information  (e.g.,  technical  progress  on  one’s  projects,  other 
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agency  interests),  collating  information  (e.g.,  as  submitted  by  Pis;  from  earlier  presentations),  and 
preparing  visual  aids  from  these. 

A  final  task  category,  that  PMs  said  did  not  represent  an  unwieldy  load,  is  administrative 
overhead.  This  includes  preparing  regular  and  “fire  drill”  reports  regarding  program  status;  doing 
financial  forecasts  and  budgets;  mentoring  subordinates  and  less  experienced  managers;  and 
contributing  to  planning  and  strategy  committees. 


The  New  Manager 

The  typical  ARPA  PM  is  a  technically  sophisticated  individual  -  industrial,  academic  or 
military  officer  -  with  more  or  less  experience  in  the  DoD  R&D  environment.  A  number  of  PMs 
are  noted  scientists  and  technologists  in  their  right. 

New  PMs  accept  an  assignment  at  ARPA  for  a  variety  of  reasons  -  passion  for  a  particular 
technology,  the  opportunity  to  do  something  very  different,  the  attraction  of  high  risk  -  high  dollar 
programs,  and  others.  Those  who  hold  senior  management  positions  in  home  companies  take  a 
risk  that  the  position  will  evaporate  when  the  ARPA  assignment  is  over.  However,  those  who 
depart  with  new  managerial  skills,  a  deep  understanding  of  the  Washington  procurement  game, 
and  a  broad  base  of  contacts  in  the  advanced  technology  domain  are  highly  valued. 

Assignments  typically  last  four  years  or  less.  For  PMs,  the  years  decompose  into  three 
phases: 

1.  Becoming  familiar  enough  with  ARPA  and  the  Washington,  D.C.  procurement  venue  to 
launch  (or  assume  control  of)  a  program; 

2.  Conducting  the  program  that  led  to  the  assignment  in  the  first  place;  and 

3.  Wrapping  up  or  transferring  program  control,  and  preparing  to  return  to  the  home 
organization  or  to  a  new  opportunity. 


The  new  PM  often  needs  6  to  12  months  to  become  familiar  with  the  process,  and  even  more 
time  to  master  it.  Unfortunately,  this  duration  is  too  long  relative  to  the  2-  to  3-year  assignment 
and  puts  a  great  deal  of  time  pressure  on  the  PM  to  have  significant  accomplishments  before 
having  to  wrap  up  and  head  home.  While  the  phases  may  overlap,  one  key  observation  is  that  the 
knowledge  and  skills  that  PMs  need  to  succeed  in  this  environment  are  complex  and  cannot  be 
learned  and  assimilated  instantaneously.  Thus,  advance  preparation  may  be  helpful. 

Upon  arrival  at  ARPA,  new  PMs  receive  no  formal  “instruction”  in  how  ARPA  works  or  how 
to  work  ARPA.  The  fortunate  new  PMs  are  taken  into  a  mentoring  relationship,  often  with  an 
individual  who  has  been  instrumental  in  attracting  them  to  ARPA  in  the  first  place.  In  some  cases, 
new  PMs  have  managed  only  projects  one  or  two  orders  of  magnitude  smaller  (in  terms  of 
dollars)  than  those  that  they  inherit  or  will  initiate  while  at  ARPA.  Obviously,  this  carries  a 
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variety  of  risks  -  financial  risks  of  misspending  significant  resources  and  the  risk  of  losing 
sponsors’  interest  if  the  program  produces  little  of  value. 

In  general,  the  PMs  we  spoke  with  said  that  the  mentoring  process  work  pretty  well,  but 
suffers  from  two  key  flaws.  First,  it  is  expensive.  When  mentoring  begins  to  absorb  a  good  deal 
of  time,  two  people  -  the  learner  and  the  mentor  -  are  tied  up,  which  reduces  overall  productivity. 
An  interesting  metric  that  emerges  in  this  regard,  that  could  be  used  to  judge  the  merit  of  a 
support  system,  is  “reclamation  of  human  resources.”  This  refers  to  time  devoted  to  mentoring,  a 
measure  that  an  activity-based  cost  accounting  system  could  be  used  to  calculate,  i.e.,  distribution 
of  managers’  time  with  or  without  a  support  system. 

Second,  informal  mentoring  has  no  quality  control  mechanism.  The  system  cannot  easily 
ensure  that  different  mentors  teach  different  new  managers  similar  “survival  skills,”  approaches, 
values,  etc.  Moreover,  each  new  mentor-manager  pair  provides  another  opportunity  for  mutation 
in  ARPA’s  operating  procedures.  Not  that  this  is  inherently  bad,  but  it  does  make  for  inefficient 
use  of  a  PM’s  time  to  invent  procedures  to  get  programs  accomplished. 

Clearly,  ARPA  would  like  to  have  new  PMs  behave  like  seasoned  managers  right  from  the 
start,  and  to  invest  the  minimum  resources  to  attain  this  goal.  The  knowledge  and  skills  that 
managers  need,  however,  are  complex  and  cannot  be  learned  instantaneously.  New  managers 
must  quickly  take  up  responsibility  for  a  program,  sometimes  one  that  is  already  in  progress. 

Since  SOPs  at  ARPA  are  relatively  limited,  once  involved  in  a  program,  PMs  depend  a  great  deal 
on  being  coached  by  other  program  managers.  There  seems  to  be  good  cooperation  among  PMs 
in  this  regard.  However,  the  logistics  of  this  arrangement  may  provide  significant  opportunities  to 
improve  productivity,  lower  risks  and  increase  program  quality. 


A  FRAMEWORK  FOR  AIDING  THE  R&D  MANAGER 


Overall  System  Goals 

Based  on  the  above  insights  into  the  ARPA  PM’s  job,  we  identified  five  goals  that  should 
guide  development  of  a  PM’s  support  system. 


1.  Enhanced  program  productivity  with  greater  adoption  of  ARPA  project  outputs  in  DoD 
and  civilian  enterprises. 

2.  Predictable,  well-controlled  and  repeatable  program  management  process,  independent  of 
program  manager  (i.e.,  experienced  or  new),  area  (SSTO,  ESTO,  etc.),  and  type  of 
project. 

3.  Reduced  complexity  of  the  manager’s  job  to  improve  productivity  and  reduce  learning 
requirements. 

4.  Enhanced  performance  by  the  manager  in  both  technical  and  administrative  tasks. 
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5.  Ability  to  bring  new  managers  up  to  speed  more  rapidly  and  at  lower  cost. 

We  are  convinced  that  a  PMA  can  help  ARPA  progress  toward  these  goals.  Such  a 
computer-based  support  system.  The  following  sections  describe  our  structure  for  a  PMA. 

To  have  broad  commercial  appeal,  similar  benefits  should  accrue  to  R&D  managers  in 
enterprises  other  than  ARPA.  These  include  other  government  agencies  (e.g.,  DoD,  DoE,  DoT, 
etc.),  R&D  consortia  (e.g.,  SEMATECH,  MCC),  industrial  laboratories,  and  university  labs. 
Based  on  the  number  of  such  institutions  (thousands)  and  R&D  planners  per  institution  (tens  to 
hundreds)  who  might  benefit  from  a  PM  support  system,  a  conservative  estimate  of  the  potential 
market  for  such  a  system  is  on  the  order  of  100,000  units. 


A  Framework  for  R&D 

Figure  2  depicts  an  idealized  process  view  of  the  PM’s  major  tasks.  Ellipses  indicate  the  major 
steps  and  the  other  elements  in  the  figure  indicate  inputs  and  outputs  of  these  steps.  Briefly,  in 
needs  definition,  the  PM  interacts  with  potential  technology  users  to  develop  a  clear 
understanding  of  what  they  will  value  and  in  what  time  frame.  This  output  is  expressed  as  a  set  of 
relationships  between  attribute  changes  (e.g.,  speed,  maintainability,  etc.),  value,  and  time. 

Technology  mapping  uses  the  characterization  of  customer  desires  along  with  technology 
monitoring,  forecasts  and  assessments  to  produce  alternative  technological  approaches  (A,  B, 
C...).  These  approaches  harbor  various  levels  of  risk.  To  determine  these  risks  more  accurately, 
the  PM  decomposes  each  potential  solution  into  its  component  requirements  (tree  structure),  and 
measures  or  estimates  risk  at  the  component  level.  Risk  is  assessed  along  several  dimensions 
including  the  market,  scientific/technological,  realization  (manufacturing)  and  organizational  tool. 

Program  roadmap  construction  allows  the  PM  to  explore  benefit/risk  tradeoffs  by  comparing 
alternative  sets  of  projects,  resource  levels,  and  temporal  orderings.  PMA  will  offer 
computational  support  to  facilitate  “what  if’  analyses.  It  will  enable  the  PM  to  visualize 
alternative  formulations  in  terms  of  resources,  time  required,  and  relative  risks.  PMA  also  allows 
the  PM  to  weave  in  recognition  of  R&D  efforts  being  supported  by  other  ARPA  and  outside 
groups  to  leverage  their  advances  and  buttress  prospects  for  success  should  certain  components 
not  succeed. 

PMA  supports  program  management  via  two  important  applications  -  presentation 
preparation  and  program  execution  and  management.  One  of  the  key  features  of  PMA  is  its 
combination  of  program  planning  with  program  execution.  Presentation  preparation  helps  the  PM 
generate  and  present  persuasive  materials  to  communicate  with  various  audiences  (e.g.,  potential 
sponsors  and  supporters).  As  part  of  PMA  this  gains  in  accessing  the  other  components,  such  as 
the  program  roadmap  to  show  how  component  projects  fit  with  each  other  and  with  others’  R&D. 
Most  practically,  it  will  facilitate  access  to  and  reuse  of  information  as  the  PM  tailors  particular 
presentations. 
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Program  execution  and  management  will  likely  account  for  most  of  the  day-to-day  use  of 
PMA.  It  helps  the  PM  accomplish  project  portfolio  management  by  supporting  information 
gathering  (e.g.,  orchestrating  e-mail  queries  to  one’s  Pis;  compiling  information  semiautomatically 
from  PI  home  pages),  scheduling  (e.g.,  calendar  tracking  of  project  milestones;  meeting 
scheduling),  budgeting  (e.g.,  tracking  program  expenditures;  facilitating  reallocations),  and 
evaluation  (e.g.,  recording  milestone  accomplishments  and  relating  these  to  program  and  national 
goals). 


Figure  2.  A  framework  for  technology  development 


SUPPORT  REQUIREMENTS 

Figure  2  depicts  the  overall  PMA  framework.  Table  1  lists  specific  tasks  and 
corresponding  PMA  tools  to  assist  in  the  accomplishment  of  those  tasks.  We  briefly  summarize 
these  here. 

Table  1  organizes  PM  tasks  into  the  five  main  functions  addressed  by  PMA  (cf.,  previous 
section  and  Figure  2).  We  conceive  of  the  PM  shifting  emphases  among  these  as  his  or  her 
program  matures.  In  program  development,  emphasis  will  be  on  planning,  involving  Functions  1- 
4.  During  program  management,  Functions  3-5  will  likely  predominate.  We  envision  creating 
specific  cases  that  play  out  across  these  functions  to  help  new  PMs  familiarize  themselves  with 
ARPA. 
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Table  1,  Column  1,  lists  a  number  of  tasks  within  the  five  Functions.  In  Column  2,  it  lists 
specific  tools.  Tools  do  not  neatly  map  one-to-one  against  either  tasks  or  functions,  but  are 
grouped  by  the  functions  they  most  directly  serve.  The  demonstration  PMA  shows  how  these 
tools  will  operate. 

Table  1.  Functional  requirements  for  the  program  manager’s  associate  (PMA) 


Functions/Tasks: 


Needs  Identification  and  Prioritization 


•  needs  stimulation 

•  cross-program  information  sharing 

•  supporting  ARPA  Director  program 
coordination 

•  providing  rationale  for  support  from  multiple 
sources 

•  stakeholder  identification  and  coordination 


Technology  Mappin 


•  technology  forecasting  and  assessment 

•  synthesis  of  needs  and  technology 
requirements 

Technology  Decomposition 


•  depicting  technological  dependencies 

•  depicting  probabilities  of  attainment  and 
payoffs 

•  characterizing  resource  and  time 
requirements 


•  graphical  depiction  of  project  relationships 

•  capability  to  perform  “what  if’  analyses 

•  program  evaluation  support 


5.  I  Program  Management 


•  contracting  support 

•  proposal  evaluation  support 

•  project  budgeting  (with  ‘what  if 
capabilities) 

•  project  tracking  (milestone  alerting) 

•  communication  support 

•  schedule  management 

•  portfolio  assessment 

•  interact  with  the  roadmap 

•  financials 


Tools: 


•  technology  updates 

•  planning  tools 


•  attributes  X  Values  X  Time  analyses 
(changes  propagate  among  these) 

•  stakeholder  management 


•  ‘find  out  about’  agents 

•  technology  opportunities  analysis 

•  technology  forecasting 

•  technology  X  needs  analyses 


•  tree  diagrams 

•  technology  assessment  analyses 


visual  spreadsheet 
graphical  benefit/risk  metrics 
object  oriented  embedded  information 
Wizard 


financial  management  tool 
project  status  tracker 
calendaring  agent 
document  preparation  support 
evaluation  support 
communicator 
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Tools  especially  serving  needs  assessment  include: 

•  technology  updates  -  compiling  program  achievements  with  external  information  acquisition 
(using  the  “find  out  about”  tools)  to  provide  news  to  potential  sponsors,  users,  and  Pis; 
serving  to  increase  awareness  of  and  stimulate  interest  in  ARPA  program  interests  by 
providing  updates  to  a  contacts  list  electronically 

•  planning  tools  -  tailoring  Search  Technology’s  commercial  software  packages’  (Business  . 
Planning  Advisor  and  Product  Planning  Advisor)  capabilities  to  the  PM”s  domain  of  interests 

•  attributes  X  values  X  time  analyses  -  providing  computational  support  to  investigate  tradeoffs 
among  program  elements,  and  plots  to  communicate  the  intended  program  payoffs 

•  stakeholder  management  -  underlying  PMA  will  be  several  relational  databases  (e.g., 
stakeholders,  technologies)  with  tools  to  ease  recall  of  information  according  to  PM-selected 
markers;  for  instance,  the  PM  can  establish  specialty  networks  of  contacts  who  share  interests 
in  certain  technologies 


Tools  designed  particularly  for  technology  mapping  include: 

•  “FOA”  agents  -  intelligent  information  retrieval  to  ‘find  out  about’  topics  of  interest  by 
searching  internal  and  public  databases,  the  World  Wide  Web,  etc. 

•  technology  opportunities  analysis  -  programs  that  summarize  and  graph  FOA  information  to 
help  track  emerging  technologies  and  potential  applications 

•  technology  forecasting  -  additional  tools  to  facilitate  situation  assessment  and  projection 

•  technology  X  needs  analyses  -  means  to  map  functional  needs  into  technological  requirements 

Tools  for  technology  decomposition: 

•  tree  diagrams  -  to  compare  R&D  options  graphically 

•  technology  assessment  analyses  -  attendant  analyses  framed  to  provide  multiple  perspectives 
(e.g.,  value  comparisons,  technical  feasibility  vs.  risk  assessments,  market  comparisons) 

Tools  for  program  roadmapping: 

•  visual  spreadsheet  -  allowing  easy  “big  picture”  comparisons  for  program  design  (and  also 
project  tracking) 
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•  graphical  benefit/risk  metrics  -  as  spreadsheet  entries  are  changed,  plots  and  meters  reflect  the 
budgetary  and  risk-reduction  implications  over  time 

•  object  oriented  embedded  information  -  “information  at  the  PM’s  fingertips”  -  ‘pointing  and 
clicking’  on  entries  calls  up  more  detailed  information  [e.g.,  opening  a  project  gets  to  project 
summary  information;  further  opening  gets  to  contact  information  (e.g.,  Pi’s  e-mail)]; 

•  Wizard  provides  help  at  different  levels  -  simple  PMA  features  help;  substantive  advice 
cumulated  over  related  ARPA  experience. 


Tools  for  program  management: 

•  financial  management  tool  -  with  spreadsheet  and  graphical  budgeting  capabilities  enabling 
various  analyses,  with  cross-application  information  updating 

•  project  status  tracker:  means  to  determine  contractors’  technical  and  budget  status 

•  calendaring  agent 

•  document  preparation  support:  support  for  construction  of  RFIs,  RFPs,  SOW,s  and  contracts 

•  evaluation  support:  system  to  facilitate  proposals  and  to  support  program  evaluation 

•  presentation  Wizard:  tool  to  help  build  customized  presentations  drawing  upon  the  PM’s 
archive  of  graphics,  demonstrations,  etc.,  utilizing  standard  presentation  graphics  package(s) 


Application 

The  PMA  tools  interlink  to  serve  multiple  PM  functional  needs.  A  premise  of  the  design  of 
the  PMA  is  that  an  application  that  integrates  these  tools  with  each  other  and  with  standard 
software  supports  (e.g.,  word  processing,  calendaring,  spreadsheets,  presentation  graphics)  will 
boost  PM  effectiveness. 

The  PM  manager  is  the  hub  of  a  large  number  of  information  flows.  These  flows  involve 
different  types  of  artifacts,  are  more  and  less  active  as  a  function  of  project  stage,  and  exhibit 
different  volumes,  predictability,  and  priority.  “Artifacts”  that  move  along  these  channels  refer  to 
myriad  reports,  presentations,  teleconferences,  and  e-mail  that  keep  people  informed. 

Despite  the  heavy  use  of  e-mail,  the  PM  spends  a  lot  of  time  interacting  in  person  with  people. 
Similarly,  since  feedback  on  mission-critical  items,  like  equipment  purchases,  supplier  invoices, 
etc.,  is  not  automatic,  the  manager  must  expand  the  network  once  the  project  is  underway. 
Artifacts  are  exchanged  in  paper  form  and  get  lost  from  time  to  time.  They  may  also  need  to  pass 
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through  several  hands  (links  not  shown  in  Figure  2)  and  can  become  stalled  along  the  way.  When 
this  occurs,  the  manager  must  trace  the  causes  by  contacting  people  along  the  chain. 

The  manager  may  need  help  creating  and  sending  original  reports,  but  may  need  more  help  in 
reusing  information  to  service  different  channels  or  the  same  channel  over  time.  All  managers 
said  that  they  spend  too  much  time  producing  minor  variations  of  similar  reports  for  different 
people.  For  instance,  presentation  vugraphs  of  similar  information,  but  different  format,  are  used 
for  presentation  to  several  groups.  Each  set  has  a  relatively  short  life,  yet  consumes  much  time  to 
produce. 

Figure  2  helps  to  explain  why  new  assignees  have  difficulty  becoming  effective  -  too  many 
undocumented  channels.  The  new  manager  must  learn  the  identity  of  each  channel,  how  to 
manage  its  idiosyncratic  flow,  why  it  is  important,  its  authority,  what  other  areas  link  to  it,  etc. 

The  manager  acquires  some  of  this  knowledge  from  peers  and  managers.  He  or  she  learns  more 
about  how  the  system  works  when,  unfortunately,  information  gets  lost  or  stalled  and  must  be 
sleuthed  down.  The  PMA  offers  special  promise  as  a  mechanism  to  enrich  and  hasten  PM 
learning  about  the  job  and  proficiency  at  it. 

Figure  3  portrays  the  general  PMA  architecture.  PMA  is  presently  designed  as  a  Windows 
application.  One  option  is  to  build  it  using  authoring  systems  that  enable  easy  porting  to  Unix  and 
Macintosh  environments.  Within  Windows,  PMA  appears  as  a  discrete  application.  It  uses  a 
recognizable  interface  format  with  its  arrays  of  tools.  Underlying  PMA  are  specific  objects  with 
appropriate  functionality  associated  (e.g.,  project  objects  are  illustrated  in  Figure  4).  PMA 
includes  protocols  to  access  other  databases  (to  obtain  information  such  as  ARPA  formats  for 
statements  of  work,  illustrated).  Full  application  of  PMA  would  entail  coordination  among  key 
information  sources  (e.g.,  accounting)  to  assure  access  when  needed.  Lastly,  note  that  PMA 
works  with  standard  software,  such  as  Microsoft  Office,  to  produce  particular  outputs. 

Document  templates  facilitate  common  format  within  ARPA  and  make  use  of  spreadsheets,  etc., 
easier. 

Development  Strategy 

Further  development  depends  upon  continued  support.  Phase  II  STTR  support  would  enable 
development  of  an  operational,  basic  PMA.  This  should  be  seen  as  the  next  step  in  developing  a 
more  advanced  PMA.  The  basic  PMA  will  be  developed  in  accord  with  Figures  2-4  and  Table  1 . 
The  first  stage  would  be  to  seek  feedback  on  most  desired  features  and  requirements  for  adoption 
by  showing  the  demonstration  to  a  wide  spectrum  of  ARPA  Pms.  The  second  stage  will  be  to 
determine  system  development  parameters  (e.g.,  whether  it  should  operate  under  multiple 
systems).  Then  the  basic  PMA  engine  will  be  prepared.  Fourth,  specific  tools  will  be  constructed 
(cf..  Table  1).  We  anticipate  that  functional  versions  of  nearly  all  the  tools  listed  in  Table  1  should 
be  available  by  the  completion  of  Phase  II. 

Development  beyond  Phase  II  will  first  entail  improvement  for  ARPA  based  on  feedback  from 
lead  users.  Then  it  will  involve  functional  improvement  of  the  tools  (e.g.,  increasing  the 
intelligence  of  the  search  mechanisms,  increasing  options  within  the  tools,  offering  support  for 
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links  to  other  software  applications).  One  of  the  most  exciting  dimensions  will  be  to  enhance  the 
learning  capabilities  of  PMA  so  that  it  can  adapt  to  the  needs  of  individual  Pms  and  improve  its 
performance  for  each  PM  with  practice. 

We  also  keep  in  mind  that  an  objective  of  this  STTR  program  is  commercialization.  We  will 
continue  to  work  to  develop  versions  of  PMA  suited  to  R&D  managers  outside  ARPA. 


Figure  3.  An  object-based  architecture  for  PMA  that  uses  standard  office 

SOFTWARE  TO  AUTHOR  AND  VIEW  THE  OBJECT  DATA 


16 


Copies: 


Commander  (2  copies) 

U.S.  Army  Missile  Command 
ATTN:  AMSMI-RD-WS-DP-SB 
(Mr.  Phillips,  Tech  Monitor) 

Bldg  7770,  Room  101B 
Redstone  Arsenal,  AL  35898-5248 

Commander 

U.S.  Army  Missile  Command 
ATTN:  AMSMI-RD-CS-R 
Redstone  Arsenal,  AL  35898-5240 

Defense  Intelligence  Agency 
Missile  &  Space  Intelligence  Center 
ATTN:  MSC-2D 

Redstone  Arsenal,  AL  35898-5500 
Director 

U.S.  Army  Missile  Command 
ATTN:  AMSMI-RD-WS 
Redstone  Arsenal,  AL  35898-5248 

Director 

Advanced  Research  Projects  Agency 
ATTN:  Dr.  Craig  Wier  (SSTO) 

3701  North  Fairfax  Drive 
Arlington,  VA  22203-1714 

i/^Director 

Advanced  Research  Projects  Agency 
ATTN:  OASB/ARPA  Library 
3701  North  Fairfax  Drive 
Arlington,  VA  22203-1714 

Defense  Technical  Information  Center  (2  copies) 
ATTN:  Acquisitions/OCP 
Cameron  Station,  Bldg  5 
Alexandria,  VA  22034-6145 


